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1 Introduction 

In this paper, U(n, K) is the unitary group over K = R, C or H (the set of 
real quaternions). 

Let U be distributed with the Haar measure on U(n, C). The random 
variable det(Id„ — U) has played a crucial role in recent years in the study of 
some connections between random matrix theory and analytic number theory 
(see [H] for more details). In [TO], the authors show that det(Id n — U) can be 
decomposed as a product of n independent random variables: 

n 

det(Id„ - U) ^ Hi 1 ^ V^k~t) , (1) 

k=l 
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where u>i, . . . , w„, Bi , . . . , B\ n _x are independent, the ui' k s being uniformly 
distributed on (— tt,it) and the -Bij's (0 < j < n — 1) being beta distributed 
with parameters 1 and j (with the convention that B\ q = 1). In particular, 
from such a decomposition, fundamental quantities such as the Mellin- Fourier 
transform of det(Id„ — U) follow at once. The main ingredient to obtain the 
decomposition ([1]) is a recursive construction of the Haar measure using com- 
plex reflections. In particular, every U £ U(n, C) can be decomposed as a 
product of n independent reflections. More precisely, it is proved in [10] that 
if Si, . . . , s n are n independent random variables such that for every k < n, Sk 
is uniformly distributed on the fc-th dimensional unit sphere 5f k in C fc and if 
BP^ is the reflection of C fc mapping Sk onto the first vector of the canonical 
basis, then 

(n) /Idi \ /Id„_ 2 \ /ld n _ X \ 

R { Rl"-»)~\ o rw){ RW ) ~ tow, 

where /J-u(n,c) stands for the Haar measure on £/(n, C). At this stage two re- 
marks are in order. First, a similar method works to generate the Haar measure 
on the orthogonal group 0(n,R) (see [10]) and this was already noticed by 
Mezzadri in [25] using Householder reflections. But as already noticed in |10| . 
Householder reflections would not work for U(n, C) (see next section for more 
details). Moreover in [TO], a decomposition such as (fTJ) could not be obtained 
for the symplectic group USp(2n, C), which also plays an important role in 
the connections between random matrix theory and the study of families of L 
functions (see [19 , [20J). Indeed, there does not seem to be a natural way to 
generate recursively the Haar measure on this group. 

Question 1. Is there any decomposition of det(Id„ — U) as a product 
of independent variables of the type ([T]), when U is drawn from USp(2n,C), 
according to the Haar measure? 

In this paper we shall prove that, in a sense to be made precise, if a 
subgroup Q of U (n, K) contains enough reflections, then one can recursively 
generate the Haar measure and obtain a decomposition of the type ([T]) for 
det(Id„ — [/), U € Q. In particular this will apply to U(n,M) which can be 
identified with the symplectic group, hence answering question 1 above. Our 
recursive decomposition of the Haar measure also applies to the symmetric 
group. This leads us to our second remark concerning the generation of the 
Haar measure obtained in [TO] and explained above. Indeed, this way of gen- 
erating an element of U (n, C) which is Haar distributed by choosing a vector 
(si, . . . , s n ) of independent variables from Sf 1 x . . . x =y n , each Si being uni- 
formly distributed, is reminiscent of the generation of a random permutation 
according to the so-called Chinese restaurant process which we briefly describe 
(see [35] for a complete treatment). Let [n] denote the set {1, • • • , n} and S n 
the symmetric group of order n. It is known that for n > 2, every permutation 
cr G S n can be decomposed in the following way: 
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cr = t„ o ■ ■ ■ o r 2 (2) 

where for k = 2,...,n, either r k is the identity or is the transposition 
(k,rrik) for some G [fc — 1]. In the first case we will say by extension that 
it is the transposition (fc,TOfe) with m& = fc. This decomposition is unique, see 
Tsilevich [36J, the lemma p. 4075. It corresponds to the Chinese restaurant 
generation of a permutation. Let us consider cycles as "tables". Integer 1 goes 
to the first table. If r 2 7^ Id, then integer 2 goes to the first table, at the left of 
1. If T2 = Id, it goes to a new table. When integers 1, . . . , k are placed, then 
fc + I goes to a new table if Tk+i = Id, and goes to the left of Tk+i(k + l) — jk+i 
if not. We get a bijection between [1] x [2] x ■ • • x [n] — > S n - It is projective 
(or consistent) in the sense that if a is in <S„+i the restriction of a to [n] is in 

Sn- 

In this setting, the number of cycles k„ of a permutation cr is the number 
of tables, i.e. the number of Id in @ i.e. 



fc«r = X>, (3) 



where £ r = l(r r = Id). For a matricial rewriting, we make a change of basis. 
Let e', = e n -j + i and let be the restriction of to [k]. Then the product 
in @ is represented by 



Id! \ /Id n _ 2 

R( n ~V ■■■{ RW 



If at each stage, the integer nik is chosen uniformly in [k], then the induced 
measure on S n is the uniform distribution denoted by fi$ n . 

Actually, one can more generally generate in this way the Ewens measure 
on S n (see Tsilevich [36] and Pitman [32]). The Ewens measure fjy\ > 0, is 
a deformation of /is n obtained by performing a change of probability measure 
or a sampling in the following way: 

/4(<7) = ^^s»- (4) 

{V)n 

To generate fi e nl one has to pick n integers mi,m2,...,m„, independently, 
from [1] x ■ • • x [n] according to the probability distribution 

Question 2. Is there an analogue of the Ewens measure on the unitary 
group U(n, C)? 



1 The other construction of a random permutation named Feller's coupling uses the 
variables in the reverse order £ n , • • • ,£1, but this construction is not projective. 
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We shall see in this paper that there indeed exists an analogue of the 
Ewens measure on U(n, C): more precisely we generalize Q to unitary groups 
and a particular class of their subgroups. The analogue of transpositions are 
reflections and the weight of the sampling is now det(Id — U) s det(Id — U) s , 
S e C, *Ke(<5) > —1/2, so that the measure f^j n \ on U(n), which is defined by 

E^ ( . (f(U) det(Id - U) 7 det(ld - U) 5 ) 

V W)) = ^7 = 

u{n) E ww (det(Id - t/)" 5 det(Id - U) s \ 

for any test function /, is the analogue of the Ewens measure. Such sam- 
plings with (5 £ K have already been studied on the finite-dimensional unitary 
group by Hua [18J, and results about the infinite dimensional case (on complex 
Grassmannians) were given by Pickrell ([3U] and [21] )• More recently, Neretin 
|27| also considered this measure, introducing the possibility 6 € C. Borodin 
and Olshanski [7] have used the analogue of this measure in the framework 
of the infinite dimensional unitary group and proved ergodic properties. For- 
rester and Witte in [35] referred to this measure as the cJUE distribution. 
We also studied this ensemble in [12] in relation with the theory of orthog- 
onal polynomials on the unit circle. Following [35] and [H] we shall call the 
ensemble of unitary matrices endowed with this sampled measure the circular 
Jacobi ensemble. 

It is natural to ask whether the circular Jacobi ensemble has some interest- 
ing properties: indeed, the case 8 — corresponds to the Haar measure and it 
is well known this ensemble enjoys many remarkable spectral properties. For 
instance, the point process associated to the eigenvalues is determinantal and 
the associated rescaled kernel converges to the sine kernel . The projection of 
the measures f^) n \ on the spectrum has the density 

™ j=l l<i<j<n 

where the weight u> T on T = {e l9 , 9 e [— tt, it}} is defined by 

w J {e ie ) = (l-c ie ) S (l-e- w ) s = (2-2cos60 Q e- b(7rsgne - 9) , 

(5 = a + ib) and Z n is a normalization constant. Note that when b =/= 0, 
an asymmetric singularity at 1 occurs. The statistical properties of the O^s 
depend on the successive orthonormal polynomials ((fk) with respect to the 
normalized version {« T of w T and the normalized reproducing kernel 

< 
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In [7] the authors consider the image of Mw n ) by the Cayley transform 
on the set of Hermitian matrices and make a thorough study of the spec- 
tral properties of this random matrix ensemble. In particular they prove that 
the eigenvalues form a determinantal process and show that the associated 
rescaled kernel converges to some hypergeometric kernel. As expected, we 
shall see that the eigenvalues process of the circular Jacobi ensemble is also 
determinantal and for every n, we identify the hypergeometric kernel Kn 
associated with it. 

Question 3. Is there an appropriate rescaling of the kernels Kn such 
that the rescaled kernels converge to some kernel K^l 

We shall see that the answer to question 3 is positive and that the kernel 
K^} is a confluent hypergeometric kernel, with a natural connection to that 
obtained by Borodin and Olshanski in [7] on the set of Hermitian matrices. 
The case 8 = corresponds to the sine kernel. 

The weight w T is a generic example leading to a singularity 

c^\e\ 2a t e>0 + c^\9\ 2a t e<0 

at 8 = 0, with distinct positive constants c^ + ' and c^~\ The confluent hyperge- 
ometric kernel, depending on the two parameters a and b = ^ log(c( _ ) /c^'), 
is actually universal for the measures presenting the above singularity, as 
proved in a forthcoming paper, following the method initiated by Lubinsky 
(|22|. |23|V For a universality result when S is real see [33J. 

The layout of the paper is as follows. In Section[2]we present the generation 
by reflections and deduce a splitting formula for the characteristic polynomial 
(Theorem [2]). As an application, we define the generalized Ewens measure 
depending on the complex parameter 5 (Theorem [3]). Section [3] is devoted to 
a study of the kernel which governs the correlations of eigenvalues when the 
unitary group is equipped with this measure and its asymptotics (Theorem 
[5|. The main properties of the families of hypergeometric functions 2-F1 and 
1-F1 are recalled in the Appendix. 

2 Generating the Haar measure and the generalized 
Ewens measure 

2.1 Complex reflections 

Reflections play a central role in the generation of the Haar measure for the 
classical compact groups. In the case of 0(n) the decomposition into a product 
of reflections is well known, see [TS] and other references as explained in [33] . 
Householder reflections are generally used in the case of 0(n), but they are 
not suitable for U(n,C). Indeed, recall that Householder reflections are of the 
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form H v = Id — 2v(v\ •). For every unit y, it is possible to choose v such that 
H v y — ae,i with a = ±y*^-| , where e\ is the first element of the canonical basis. 
So when the ground field is C, then a ^ 1 in general and there does not exist 
a Householder reflection which maps y onto ei, whereas this can always be 
achieved when the ground field is M. That is why it is not possible to directly 
extend the arguments in [25] to C/(n,C). In [10] and [12] it is proposed to 
use complex (resp. quaternionic) proper reflections, that is norm preserving 
automorphisms of C" (resp. H") that leave exactly one hyperplane pointwise 
fixed. So a reflection will be either the identity or a unitary transformation U 
such that / — U is of rank one. It may be written as 

(1-A)(a,y) 

s a ,x(y)=y-a K - — pg-i^ 
m 

where a £ H™ and A € H with |A| = 1 (A is the second eigenvalue). If a; =t e±, 
there exists a reflection mapping e\ onto x. It is enough to take a = e\ — x 
and A = —(1 — xi)(l — ii)" 1 where x\ = (e\,x). 

2.2 Generating the Haar measure on U(n, K) and on some of its 
subgroups 

We first give conditions under which an element of a subgroup of U (n, K) 
(under the Haar measure) can be generated as a product of independent re- 
flections. This will lead to some remarkable identities for the characteristic 
polynomial. 

Let (ei, . . . , e n ) be an orthonormal basis of K n . Let Q be a subgroup of 
U{n, K) and for all 1 < k < n - 1, let 

H k = {G e Q | G{ej) = e jt l<j<k}, 

the subgroup of Q which stabilizes e%, • • • , et- We set 'Ho = Q- For a generic 
compact group A, we write \xj^ for the unique Haar probability measure on 
A. Finally for all 1 < k < n let pk be the map U H> U{ek)- 

Proposition 1. Let G € Q and H £ "Hi be independent random matrices, 
and assume that H ~ . Then GH ~ fig if and only if G{e\) ~ pi(fig). 

Proof. The proof is exactly the same as in [TU] Prop. 2.1, changing U{n + 1) 
into Q and U(n) into T~L. 

Definition 1. A sequence (z^o, . . . , v n -i) of probability measures on Q is said 
to be coherent with /ig if for all < k < n— 1, 

Vk{Hk) = 1 and p k+1 (v k ) = p k+ i(pL Uk ) . 

In the following, Uo * v\ v n ~\ stands for the law of a random variable 

H{)H\ . . . H n -\ where all H^s are independent and Hi ~ vi. Now we can 
provide a general method to generate an element of Q endowed with its Haar 
measure. 
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Theorem 1. If Q is a subgroup of U{n, K) and (vq, • ■ • , v n — l) * s a sequence 
of coherent measures with fig, then we have: 

tig = va *vi * ■ ■ --kVn-x. 
Proof. It is sufficient to prove by induction on 1 < k < n that 

fn-k * ^n-fc+l * ■ ■ ■ * Vn-l = i 

which gives the desired result for k = n. If k = 1 this is obvious. If the result 
is true at rank k, it remains true at rank k + 1 by a direct application of 
Proposition [1] to the groups H n -k-i and its subgroup "H ra _fc. 

As an example, take the orthogonal group 0(n). Let be the unit 

sphere {i e 1* |af| = 1} and, for s k G , let be the matrix of the 
reflection which transforms s k into e±. If Sk is uniformly distributed on 
and if all the Sk are independent, then by Theorem [TJ the matrix 

(n) (I \ /Id„_ 2 \ /Id n _ a \ 
H {0R^)---{ RW){ RW)- 

is Ho(n) distributed. 

2.3 Splitting of the characteristic polynomial 

In view to phrase a general version of formula ([1]) which is proved in |10| , we 
need the following definition: 

Definition 2. Note IZk the set of elements in T-Lk which are reflections. If for 
allO<k<n-l 

{R(e k+1 ) \Re1l k } = {H(e k+1 ) \ H e H k }, 

the group Q will be said to satisfy condition (R) (R standing for reflection). 

Remark 1. It is easy to see that U(n,K) and S n satisfy condition (R). In the 
next subsection we shall see more examples. 

Lemma 1. Let Q be a subgroup of U(n, K) which satisfies condition (R). Let 
G £ Q . Then there exist reflections R k G TZ k , < k < n — 1, such that 

G = RqRi ■ ■ ■ Rn-i- (5) 

Proof. This result has been established in [12] when Q = U(n,C). The proof 
in this more general case goes exactly along the same line. 

The following deterministic lemma is a key result to obtain a decomposition 
of det(Id„ — U) as a product of independent random variables: 
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Lemma 2. If for k = 1, . . . , n — 1, Rk G IZk, then 

n-l 

det(Id„ - i? • • • i?„-i) = J] ( X (c*+i,-Rfc(e/b+i)) ■ (6) 

fc=0 

Proof. We start with dct(Id„ - RH) = (detiJ) det(F* - R). Since H (hence 
iJ*), stabilizes ei, we have 

i) (H* - R)(ex) = ei - i?(e x ) =: a (say), 

ii) for it) _L e\, H*(w) _L ei and since R is a reflection, R(w) — w is a scalar 
multiple of a. 

By the multilinearity of the determinant, we get 

det(iP — R) = (ei, ei - fl(e x )) det(7r(iP) - Id n _i) 

which yields 

det(Id„ - RH) = (1 - ( ei ,i2(ei))) det(Id„_ x - tt(JJ)) . 

Iterating, we can conclude. □ 

The following result now follows immediately from Theorem[T]and Lemmas 
Q] and H 

Theorem 2. Let Q be a subgroup ofU(n,K) satisfying condition (R), and let 
(i^o, . . . , v n -i) be coherent with fxg. If G ~ fxg , then 

det(Id - G) '= [] (1 (c*+i. fffc(efc+i), » • 

fe=0 

where Hk ~ v\t, < k < n — 1, are independent. 

2.4 Applications 

The symmetric group. 

Consider now 5„ the group of permutations of size n. An element tr S S n can 
be identified with the matrix (^pKijXn (5 is Kronecker's symbol). It is 
clear that 1 is eigenvalue of this matrix, with eigenvector e\ + • • • + e n . Ben 
Hambly et al. IY] considered the characteristic polynomial at s ^ 1. To make 
relevant our problem of determinant splitting, we introduce wreath products, 
following the definition of Wieand |37| . 

Let F be a subgroup of T = {i G C | |a;| 2 = l}, endowed with the Haar 
probability measure fip . Then the wreath product F I S n provides another 
example of determinant-splitting. An element of F n can be thought of as 
a function from the set [n] to F. The group S n acts on F n in the fol- 
lowing way: if / = (/(l), . . . , /(»)) € F" and a E S n , define f a e F n 
to be the function f a = f o a . Finally take the product on F n to be 
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(/(l), . . . , f(n)) ■ (g(l), . . .,g(n)) = (fg(l), fg(n)). The wreath product of 
F by S n , denoted F lS n , is the group of elements {(/; s) : / G F n , a £ S n } 
with multiplication 

(f;a) ■ (h;<r') = (fh a ;aa') . 

If we represent (/; a) by the matrix (/(i)^-(j))i<i,.j<n> then the product in 
FiS n corresponds to the usual matricial product which makes FiS n a subgroup 
of U(n, C). The usual examples are F = {1}, F = Z 2 and F = T. 

Corollary 1. Let G € (?(= FlS n ) be \xg distributed. Then 

n 

det(Id„-G) ^ JJ(l- ei X,), 

j'=i 

urei/i Si, . . . ,e n , X\, . . . , X n independent random variables, the Ej 's fip dis- 
tributed, P(Xj = j) = ¥(Xj = 0) = 1 - 

Proof. We apply Theorem [3J As reflections correspond now to transpositions, 
condition (R) holds. Moreover Rk(ek+\) is uniformly distributed on the set 
Fek+i U • • ■ U Fe n , so that (e^+i, Rk{ek+i)) is with probability (n — k)/n 
and otherwise, it is uniform on F. □ 

Remark 2. Notice that if G = (/; a) with <r = r„ o • • • o r 2 (cf. (|5J)), then Xj is 
the indicator function of r n -j+i = Id. 

Unitary and orthogonal groups 

Take Q = U(n,C). Then \iu k = fk(jJ-u(n-k,C)) where f k :Ae U(n - k, C) h> 
Idfc© A As all reflections with respect to a hyperplane of C n_fc are elements of 
U(n — k, C), one can apply Theorem[T]and LemmaH The Hermitian products 
(efe,/ife(efc)) are distributed as the first coordinate of the first vector of an 
element of U(n — k,C), that is to say the first coordinate of the (n — k)- 
dimensional unit complex sphere with uniform measure : 

(efc+i,-fffc(efc+i)) = e lUn Bi !n -k-i 

with lo„ uniform on (— 7r, 7r) and independent of Bx,n-k-i, a beta variable 
with parameters 1 and n — k — 1. 

Therefore, as a consequence of Theorem [3J we obtain the following decom- 
position formula derived in [ID]. For g £ U(n,C) which is Hu{n,C) distributed, 
one has 

n 

det(Id„ - G) 'i= w [] (l - e lw * /B^TT) , 
fc=i 

with u>i, . . . , lo„, B\fi, . . . , £?i.„_i independent random variables, the lu^s uni- 
formly distributed on (— tt, 7t) and the -Bij's (0 < j < n — 1) being beta 
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distributed with parameters 1 and j (by convention, Bi = 1). 

A similar reasoning may be applied to SO(2n) (with the complex unit 
spheres replaced by the real ones) to yield the following: let G £ SO(2n) be 
Hso(2n) distributed, then (Corollary 6.2 in |10| ) 

2n 

law 



det(Id 2n - G) = 2 JJ (l - 



k=2 

The quaternionic group 

Our goal with this example is to solve Question 1 which was raised in the 
Introduction. To this end we establish an analogous to Lemma [5] and use the 
fact that U(n,M) = USp(2n) which is also denoted Sp(n), see for instance 
|25) Theorem 2. Then we apply Theorem [T] Let us give details. Recall that 
the symplectic group USp(2n,C) is defined as USp(2n,C) = {U € U(2n,C) \ 
UJ^U = J„}, with 

Let 

H M(2,C) 

. , . , , f a + ib c + id 

a + w + jc + ko! h->- 

\— c + id a — w 

be the usual representation of quaternions. It is a continuous injective ring 
morphism such that (f>(x) = (f>(x)* . It induces the ring morphism 



In particular 



M(n, H) M(2n,C) 

l<ij <n 



#(C7(n, H)) = {Ge C/(2n, C) : GZ n *G = Z n ] 
where Z n = J% ® ■ ■ ■ ® Ji and J\ = \ ^ \ ) . Since Z„ is conjugate to J„, 



-1 0, 

defined by the set $(U(n,W)) is therefore conjugate to USp(2n, C). We 
can therefore consider det(I — ^(G)) 

Lemma 3. If for k = 1, . . . , n — 1, Rk € 7?-fc, ^en 



n-i 

det(Id 2 „ - <?(i? • • • R n -i)) = J] dct ( Id 2 ~ <t>((e k +i,Rk(ek+i))) ■ (8) 

fc=0 
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Proof. Let us first remark that the canonical basis e\, . . . , e n of HI™ is mapped 
by # into the canonical basis e\, . . . ,£2,1 of C 2n , where the 2n x 2 matrix 
[e-2k-\i e 2k\ is exactly <£(efc). Moreover, if R is a proper reflection (leaving 
invariant an hyperplane), <?(i?) is a bireflection of C 2n i.e. a unitary transfor- 
mation leaving invariant a vector space of codimension 2. 
We start with 

det ((Id 2n -$(RH)) = det (ld 2n -$(R)$(H)) = det$(H) dot (<P(H*)-${R)) 

Since 7J (hence H*) stabilizes e y , then <P(H) (and <!>(H)*) stabilizes ex and 
£2, so we have: 

i) (H* — R){e\) =e\ — R{e\) =: a = [01,02] (say), hence, for i = 1,2, 



ii) Assume that (ex,w) = 0. Trivially, (ei,H*(w)) = hence 4>(H*)(w) is 
a matrix whose column vectors are orthogonal to £1 and £2- Moreover, since 
R is a quaternionic reflection, R(w) — w is a (right) scalar multiple of a (see 
|14j Proposition 1.6), so <P (R{w) — id) is a 2n x 2 matrix whose columns are 
in Span (a±, 02). 

By the multilinearity of the determinant, we get 

det (#(JT)* - #(JJ)) = det«e i ,a i )i<ij< 2 )det(7r( J H'*) - id 2 „_ 2 ) 
which yields 

det(Id„ - #(iiff)) = det(Id 2 - 0«ei, JJ(ei)))) det(Id 2n -2 - n(H)) . 
Iterating, we can conclude. □ 

Corollary 2. Symplectic group. Let G E USp(2n, C) be /iusp(2n,C) dis- 
tributed. Then 



where the vectors (flu, bk, c&, dk), 1 < k < n are independent and (dfe, Ck, dk) 
are 4 coordinates of the 4k- dimensional real unit sphere endowed with the 
uniform measure. 



- «P(fl))(£i) = Ei - $(R)(£i) =: a, • 



n 



det(Id 2 „ - G) =* J] ((«* ~ I) 2 + b 2 k + c\ + 4) 



fc=i 



Remark 3. We have (a&, c/-, dfc) = 



1 



= (^1,^2,^3,^4), with the 



4ft 




p law 



— _D 



p g-p , 
2 ' 2 
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we get the somehow more tractable identity in law 

law 



det(Id 2n -G) 'i= w n 1 + 



fc=i 



,2k-i 



1 - Bi 2k-i ) S §,2fc-2 



with all variables independent, P(e / i c = 1) = P(ej. = — 1) = 1/2. 

This method can be applied to other interesting groups such as USp(2n, ] 
{u G U(2n, R) | U£ tt = z} thanks to the morphism 



i& h-> 



M(2,I 
a — & 
6 a 



The traditional representation of the quaternions in M(4,R) 



: < 



a + i6 + jc + kd i— > 



M(4,R) 
/ a —6 — c — cA 
a — d — c 
c d a —b 
yd — c b a J 



gives another identity in law for a compact subgroup of f/(4n, R). 
2.5 The generalized Ewens measure 

In this section we wish to define a generalization of the Ewens measure on 
U (n, K) and some of its subgroups which will agree with the classical definition 
on the symmetric group. We first recall the definition of the Ewens measure 
on the symmetric group and how it can be generated. 



The Ewens measure on S n 

Recall (see ^ Section 1) that every permutation a £ S n can be decomposed 
in the following way: 

o = t„ o ■ • • o r 2 (9) 

where for k — 2, . . . , n, r^- is either the identity or the transposition (fc, m^) 
for some G [k — 1]. In the first case we will say by extension that it is the 
transposition (fc, rrik) with rrik = k. The number of cycles in the decomposition 
of a is denoted k a . The system of Ewens measures of parameter > consists 
in choosing the to^, k — 1, . . . , n independently, with distribution 



P(m fc = j) 



1 



+ k - 1 v ' 6 + k-l 

It is known that the induced probability on S n is 



,j = l, 



1. 



AW) 



(0)n 



(10) 
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The generalized Ewens measure 

In the following, Q is any subgroup of U(n, K). Take 6 € C such that 

< E MS (det(Id n - G) 3 "det(Id„ - G) s ^j < oo. (11) 
For < fc < n — 1 we note 



,(fe) 



Q 



6XP5 ; \G» (l-(e k+1 ,G(e k+1 ))) s (l-(e k+1 ,G(e k+1 ))) s 
Moreover, define det^ as the function 



detA 



G i ^ det(Id„ - G) s det(Id„ - G) 5 



Then the following generalization of Theorem [T] (which corresponds to the 
case (5 = 0) holds. However, note that, contrary to Theorem [TJ in the fol- 
lowing result we need that the coherent measures be supported by the set of 
reflections. 

Theorem 3. Generalized Ewens sampling formula. Let Q be a subgroup 
of U(n, K) checking condition (R) and ill}) . Let (i/q, . . . , u n -x) be a sequence 
of measures coherent with /ig, with i> k (lZ k ) = 1. We note fig^ the detg- 
sampling of fig and ujp the exp^ -sampling of v k . Then 

IS) (<S) (<5) (8) 

i.e., for all test functions f on Q , 

E M6 (/(G) det(Id„ - G)*det(Id„ - G) 5 ) 

^,/ s K---±,A^ (/(-Ro-Ri • • • Rn-i)) ~ 



E Me (det(Id n - G) 5 det(Id„ - G) 

Proof. From Theorem [2 G = Rq . . . R„-i, hence 

E M6 (/(G) det(Id„ - G) J det(Id n - G) 5 ) = 
Ei/ *...«/„_ j (/(i? • • ■ Rn-i) det(Id„ - i? . . . i?„_i) I det(Id„ - R . . . R n -i 
From LemmaEl det(Id„ - i? ■ • ■ #n-i) = IIfc=o(l _ ( e fc+i> -Rfc(efe+i)}), hence 
Ej^j *...*«/„_! (/(i2o • • ■ Rn-i) det(Id„ - i? • • • i?„-i) I det(Id„ - R . . . 

E„ Q *...*„ n _j ( /(i2o • .-ik-i) II expf '(i? fc ) ) . 



By the definition of the measures v k , this is the desired result. □ 
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Before exploring properties of this measure, let us give two examples of 
(5-samplings. 

First we check that we can recover the classical Ewens measure on the 
symmetric group. Consider Q — Z2 I S n . For S > 0, the (5-sampling in Z2 I S n 
induces a 9 = 2 2d ~ 1 sampling on S n . 

Proposition 2. For S > 0, the pushforward of fJ>^p lS by the projection 
(/, a)^ a is n e n with 9 = 2 25 " 1 . 

Similarly, if we associate with each transposition of the decomposition ^ a 
Rademacher variable, we get easily a sequence of reflections, and if denotes 
the fc-th corresponding measure, then the system (z^o, ■ ■ • , v n -i) IS coherent 
with nz 2 is n - The pushforward of v^' under the projection is a transposition 
biased by 9, so we recover the Ewens sampling formula. 

Proof. Recall that the generic element of Z2 I S n is denoted (/, a). Let C(a) 
the set of cycles of cr. If c = (d±, . . . ,dj) is such a cycle, let 1(c) = j and 
w(f; c) — Yii f{dj)- Then it is clear that 



det(xld n -(f;a)) = ]J (x e ^-w(f;c 



and in particular, 

i , /t 1 1 r w f if 3c e C(a) : w(f; c) = 1 ,, „, 

det(Id„ - (f;a)) = { ofe w „, \ ' ,. \ ' (12) 
v w ' \2 k " if VceC(a) : w(f;c) = -1. v ; 

Let P stand for /iz 2 ;s„ i- e - the uniform distribution on Z2 I S n . For any test 
function F 

E(F(a)\dct(ld n -(f,a))\ 2S ) =E[F(a)E(\det(ld n -(f,a))\ 2S \o-)] . 

Now, conditionally on cr, the weights of the cycles are independent Rademacher 
variables (i.e. ±1 with probability 1/2). So, 

P(n ceC(ff )M/,a) = -l}|a) = 2"*' 

and, due to (IT21 

E(|det(Id„-(/,cr))| 2 V) =2 (25 - 1)fc % 
which easily yields 

E ( « F(a) = [ F(a)dfx e n (a) . 

□ 
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The fundamental example remains U(n,C). In the following section, we 
will study the determinantal sructure of this model for D\z5 > —1/2. In |12j 
a precise analysis of the reflections involved in the decomposition is given. 
The case S = 1 has a specific interest. If {9\, . . . , 9 n ) are the eigenangles of a 
unitary matrix, we have 

n 

|det(Id-C/)| 2 = nil-e^| 2 , 

which, thanks to the density of the eigenangles, yields 
E (1) (f(9 1 ,...,9 n )) 

f*tr(n) 

/n 
m, . . . , 9 n ) JJ - e ie * | 2 [] |1 - e ie < | 2 d0i . . . d9 n . 

-■*'*) n j<k 1 = 1 

This means that the distribution of the eigenangles . . . , 9 n ) of a random 
matrix drawn according to M[/( n ) i s the same as the distribution of the n first 
eigenangles • • • ,9 n ) of a random matrix drawn according to (J>u(n+i,C)i 
conditionally on 9 n +i = 0, or, as seen in [38], as the distribution of {9\ — 
8 n +i, ■ ■ ■ ,9 n — 6 n+ i). More generally, in [TT], Bourgade gives a geometrical 
characterisation of this kind of measures for 8/2 £ N, defining the notion of 
conditional Haar measure. 

Remark 4- A generalized Ewens sampling formula could also be stated for 
$(G), with Q checking condition (R) and <P the ring morphism previously 
defined. 



3 A hypergeometric kernel 

In this section, we study the correlations of the point process of eigenvalues 
under the measure /itS n q and answer Question 3 (see Introduction) asked by 
Borodin-Olshanski in [7] section 8. Let us recall some basic facts on determi- 
nantal processes and correlations, referring to the books [T] 4.2 or [S] or ]W\ 
chap. 4. 

Let A = R or T = {z e C : \z\ = 1} = {c w :9 € [-7r,7r]} and let us fix 
an integer n. The collection of eigenvalues (Aj., . . . , A„) of a random n x n 
Hermitian (resp. unitary) matrix can be viewed as a point process on A , i.e. 
a random counting measure v n = 5\ 1 + • • ■ + 5\ n . Let us consider a simple 
point process v on A. If there exists a sequence of locally integrable functions 
pk such that for any mutually disjoint family of subsets D\ , . . . , of A 

I pk(xi,...,Xk)dxi...dx k 



E 



n>(A 
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then the functions p^ are called the correlation functions, or joint intensities 
of the point process. In this case, the process is said to be determinantal with 
kernel K if its correlation functions pk are given by 

fc 

Pk(xi, ■ ■ ■ ,x k ) = det K(xi,Xj). 

« ,3 = 1 

For 

^ — we denote the correlations by pk,n 

for fc < n. When the joint 

density of the eigenvalues is proportional to 

n 

Y[ W ( X k) Yl \x k -Xj\ 2 
fc=l l<j<fc<n 

for some weight w, the orthogonal poynomial method shows that the point 
process of eigenvalues is determinantal. The use of Cayley transform allows to 
connect Hermitian matrices and unitary matrices. We give a detailed descrip- 
tion of the consequence of this connection for the corresponding eigenvalue 
processes in Subsection 13. 11 and its impact on the circular Jacobi ensemble in 
Subsection 13.21 Finally, we study the asymptotic behavior in Subsection 13.31 



3.1 Determinantal processes and Cayley transform 

We follow the approach of Forrester ([16] 2.5 and 4.1.4). We start with a weight 
(positive integrable function) w T on T. The pushforward of the measure 

n^ T (e i9 I] le^-e^fd^-.-d^ 

3 = 1 l<j<k<n 

by the stereographic projection (Cayley transform) 

l-e ie 6 w 1 + iA 

A = i 7j = tan - ; c = - 

1 + & e 2 ' 1 - iA 

gives the measure 

2 ^-Q w T/l±iAiV 1 + A 2 r „ Yl |A,-A,| 2 dA 1 ...dA„. 

3 = 1 ^ J ' \<j<k<n 

We define the weight w R on R as 

2\-n„..T I 1 + 1X 



w K (x) = {l + x z )- n w 



1 — IX 



Conversely 
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If the monomials l,x,...,x n are in L 2 (w R (x)dx), then the orthogonal poly- 
nomial method gives 

1 ™ 

ZkU^) II _ X o\ 2 = det ( K n( X j, X k)) 1<jk<n 

n j=l l<j<k<n 

where Z? is a normalization constant and where 



K%{x, y) = ^w^{x)w^{y) K*(x, y) 

n-l 

*■(*,¥) = £rf(*)rf(») 



and the pf are orthonormal with respect to the measure w R (x)dx. The 
Christoffel-Darboux formula gives another expression for the kernel 

K ^(x y) = — pV ^ x ^-^ - p n-i( x )Pn(y) 
K n x — y 

where Kj is the coefficient of x^ in pj(x). In terms of the monic orthogonal 
polynomials Pq,--- ,P n , this yields 

K R (x v) - V Pe ^ Pe ^ (is) 

\\Pn_4\x-y) ■ K > 

Besides, on the unit circle, we consider the polynomials Lp g (resp. <P^) or- 
thonormal (resp. monic orthogonal) with respect to the measure w T (e ie )dO. 
and their reciprocal defined by 



<f>\(z) = z'Ml/z) , <p* t (z) = z" m {\lz). 

We have then 

^n^V') II |e i ^-e^| 2 = det(^(e^,e^)) ] 



■£ 11 ^ > 11 1 1 " \ ' >)l<j,k<n 

" j=l l<j<fc<ri 



with 



ifI( z ,() = ^(zK(c)jf^,c) 

and 

n-l 
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The Christoffel-Darboux formula is now 



(see [35] 1.12 and 3.2), or 



1-zC 



||<M 2 (l-zC) 



The kernel if* (resp. K^) rules the correlation function /?JJ m (Ai, • 
(resp. p^ m (e iei , • • • ,e i0m )) for m = l,-- - ,n. 

3.2 Our weights and their characteristics 

For the sake of simplicity we use the polygamma symbol 



(15) 



(16) 



> A m ) 



r 



a,b,--- 
c,d, - ■■ 



r(a)r(b) 



r(c)r(d)---- 

For 6 — a + ib E C with a > —1/2, we will consider two weights on (— tt,tt) 

w^(e w ) = (l-e w f(l-e- w ) s = (2 - 2 cos9) a e' b(7TSSne -^ (17) 

(18) 



*4(e iy ) = (l + c ie ) 5 (l + e - lW )° = (2 + 2cos0) a e 



-i8\6 



These are "pure" Fisher-Hartwig functions. We can go from wj to wj by the 
transform 

0h> t := -0 + 7r(sgn0) (19) 
which carries the discontinuity in 9 = to the edges ±tt, so that 

c ie = -c- ir and wj(e w ) = w^(e- iT ) . (20) 

For a > —1/2, the Fourier coefficients of w\ are known ([9] Lemma 2.1) 



1 

2^ 



;[(e ie )e- ine d6» = (-l) n r 



l + <5 + <5 
6 - n + 1,6 + n + 1 



With 



1 + <5, 1 + 6 
1 + 5 + S 



the function u^e' 61 ) = c(S)wJ(e ) is a probability density on (— w, it). For W2, 
we note that 



wj(e w )e- in6 d9 = (-1)" / «;|(e iT )e inr dr. 



Moreover we go from one system of polynomials to the other by the mapping 

z i y —z. 
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It is known from 0] p. 304 and [5] p. 31-34 that for n > the n-th or- 
thonormal polynomial with respect to wj(e ie )dd is 



= r 



5_ + n, 5 + 1 
5 + n + l, 5 



-n,S+ 1 
1 — n — 6 



; z 



with 



l*r. 



r 



<5_+ (5 + n + 1, n + 1, 5 + M + 1 
<5 + n + l,5 + n + l,<5 + £ + l 



(21) 



(22) 



(see also [TB] Prop. 4.8 in the case 5 real). With the complement formula (|4"5)) 
we get the other form 



5 + 5 + 1 + n, 6 + 1 
5 + n+l, 5 + 5 + 1 



$ n (z) = r 

In view of (|47]) and <J2TJ) we identify as 
or, using l(l8jl again 



. -n, 5 + 1 -, 



— n — <5 



5 + 5 + 1 + n,5_ + 1 
5 + n + l, 5 + 5 + 1 



-n, 5 



y 6 + 5 + l' 

Borodin and Olshanski considered the following weight on 



1 - z 



-8-8 



v$(x) = (l + ix)- 5 -"(l-ia;) 



-5—n 



(23) 



(24) 



(25) 



(26) 



Since this weight depends on n, the reference measure has only a finite set of 
moments so that there is only a finite set of orthogonal polynomials (these 
are the pseudo-Jacobi polynomials) 



Pm (x) = (x-i) m 2 F 1 



—m, 5 + n — m 
6 + 5 + 2n-2m' T 



(27) 



< a + n — i. Let us call Kf n the corresponding kernel. 



3.3 Asymptotic behavior 

For the weight wf, Borodin and Olshanski considered the (thermodynamic) 
scaling limit A i— > nX and proved ([7] Theorem 2.1) 

Theorem 4 (Borodin-Olshanski). Let <Ke 5 > 1/2. 

1. We have 

lim (sgn x sgn y) n nKf n (nx,ny) = K^(x,y) (28) 
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uniformly for x, y in compact sets o/R* x R* ; where (for x y) 



P(x) 



1 

2tt' 

2 

x 



-r 



5 + 1,5 + 1 
5 + 5+1,5 + 5 + 2 



Q(x) = - 



e 

S + 5 



i+7T 



(8 — 5) sgn x 



P(x)Q(y) - Q{x)P(y) 
x-y 

5_ 2i 
5 + 5+1' ~x~ 



5 + 1 2i 
5+5+2 ' x 



2. The limiting correlation is given by 

limn m p nm (n\ lr ■ ■ ,n\ m ) = det ( K^, (A», Xj) ) 

n \ / i<i, 



j<m 



(29) 
(30) 
(31) 

(32) 



The kernel 1(^(1 /x, 1/y) is called the confluent hypergeometric kernel in [S]. 

For the circular model, we choose the set-up wi for the sake of consistency 
with the above sections. The singularity is in z = 1 i.e. 9 = 0. To study the 
asymptotic behavior of the point process on T at the singularity (edge) we 
have two ways: either take the thermodynamic scaling 9 *-+ 6/n, or use the 
result on M. 

Theorem 5. Let V\z5> -1/2. 
1. With the weight wi, 

limn- 1 Xl> 1 (e w / n ,e ir / n )=^I(e,r) (33) 

n 

with, for 6 ^ r 



1 + 5,1 + 5 
1 + 5 + 5,1 + 5 + 5 



P T (<9)P T (r) - P t (9)P t {t) 



(34) 



where 

p*(0) : = lOl^S-W-Vwe lFl 



5 

5 + 5 + 1 
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P( - 2+ 1 ) 



(35) 



and 



Ki 



|<5+<5 



2tt 



-r 



1 + 5,1 + 5 
1 + 5 + 5, 1 + 5 + 5 



5+5+1 ■ 



|F| 1 5 + 5 + 1 < W 



2 iF x 



1 



5+5+2 ' 



;i0 



(36) 
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2. The limiting correlation is given by 

limn"^^^/", • • • ,e i( W") = det fe(^)) ■ (37) 

n \ / l<i,j<m 

Proof. We begin with a direct proof of (|33|) when (9 + 1 t, and then proceed 
with the proof of (f33|) when — t, which directly yields (j37j) and we end with 
an alternate proof of ([57)) using (|3"2")l and the Cayley transform. 

1) The following lemma describes the asymptotical behavior of the quantities 
entering in the kernel. 



Lemma 4. Wh 



en n — > oo 



Um II $„ 



r 



5 + 1,5 + 1 
5 + 5 + 1 



Moreover if n9 n —¥ 0, then (uniformly for 6 in a compact set) 

5 + 1 



limn^^*(e i9 ") = r 
limn- 5+1 (<?*)'(e ie ") = 5T 



5 + 1 
5 + 5 + 1 

5 + 1 
5 + 5 + 1 

5 + 1 
5 + 5 + 2 



iFi 
iFx 
iFx 



5 + 5 + 1 
5 

5 + 5 + 1 

5 + 1 
5 + 5 + 2 



■ \e 

■ i9 



i6 



Proof. Let us first recall that, as n — > oo, 

r(c + n) 



(38) 

(39) 
(40) 
(41) 

(42) 



which gives immediately (|38)) . The limits in ([39| and (|40|) are then conse- 
quences of (|23|) . (|25|) and the limiting relation (|50|) . Besides, in view of (|49l 
and (1251). 



(<^)'W 



-r 



5 + 5 + n,5 + l 
5 + 5 + 1,5 + n+l 



-n+ 1,(5 + 1 
(5 + <5 + 2 



(5 + 5+1 
It remains to apply ([50)1 . □ 

A) For 6 ^ t, we have, by the Christofiel-Darboux formula (fTS)) : 
lim i(0 - T )r(5 + 5 + l)n- is+S+1 ^Ky(e w/n ,e iT ^) = 



1 - z 



iFx 



5 + 5 + 1 

5 + 1 
5 + 5 + 1 



; -W) iFi 
; -iff) iFi 



5 + 5 + 1 

5 + 1 

5 + 5 + 1 
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Now, applying the Rummer's formula (|52p 

'«G#Ji'-")-'- H, '«G + l + i'» 

; iT ) = e ' T ' F '(j+!+ii - iT 

Besides we have (recall that we used Wi) 

£T,l( e «V« e W») 



c(<5) v/ wi (e i6 / n )wi (e iT /™) 



kI' 1 

and from the very definition of w\ 

limn 2 ( ,5+ ^u;i(e i9 /™)u;i(e ir/n ) = |0 T | 2 ^V 3ro57r ( ssne+8snT ) 
We conclude that holds true. 

B) On the diagonal In the following z and £ are elements of T. If F and G 
are differentiable functions on T, the de l'Hospital rule gives 

hm mm^mm = nz)G{z) _ F(Z)G > (Z) . 

C^z z - c 

Taking 



F(z) = z-"* n (z) , = *«(*) , 

so that 

F'(z) = -^-"-^(z) + z- n $' n (z) , G'(z) = -z~*®Jz) 
we get the value of the kernel on the diagonal: 



(C)-*„(*)*n(C) 



lim nv y »vsy nvsy = _ n |^ (z) |2 + 2Ste[$ n (z)z& n {z)] 

C^z 1 — z( 



= n|«^(z)| 2 -25le[*i(«)z(#;) , («)]. 

(43) 



It remains to apply the lemma. 
Notice that 



limn-^+'+^^CM) = -— L r 



2) Aiternate proof of i37p 

The pushforward of the measure 



/0*' 2 (xi, • • • ,x„)dxi . . .dx„ 
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2->«' 2 (tan|,...,tan^ 



which, at the level of kernels gives 



n 

fc=i 



cos 2 d8i . . . d9 n 



K 



tan — , tan -f , 

2 2 J 2 cos 6; cos 0,- 



Coming back to the superscript 1 with the help of we obtain 
pl%(e i6 \--- ,e i6 ™)) =det[H n (6 i ),H n (6 j )] 1 < ij < m 

with 

H n (9,9')=K^ 



cot -, — cot — . 

2' 2 J 2|sin|sin%| 

Let us rescale the angles. Since lim n ntan^- = 9 , lim n ntan^ = 9' and 
since the limit in (|28[) is uniform on compact subsets, we get 



lim — H n 

n 



\99' 



We remark that P T (9) = P(x) with x9 — —2. Moreover, from (|53|) , we 
have 

i 



6 + 6 + 1 



Q(x) = PT(0) - P T (0) 



so that, if r 
i 



-2/1/ 



5 + 5 + 1 
and consequently 

□ 



P(x)Q(y) - P(y)Q(x)\ = P T (fl)pT( T ) _ P T (r)pT(0) 
°-I-Kl(9 > r)=Kl(x,y). 



(44) 



Remark 5. 1. To have a graphical point of view of this kernel, we refer to 
[13] p.56-60. 

2. In [26], the behavior of the limiting kernel on R is used to study asymp- 
totics of the maximal eigenvalue of the generalized Cauchy ensemble. 

3. An easy computation shows that for 6 real, 6 > —1/2, we recover the 
Bessel kernel 

2(0 -r) ' 
n{9 - t) ' 



KL = ^V9t 



and for 6 = the sine kernel 
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4 Appendix: Hypergeometric functions 

For a classical reference on hypergeometric functions, see 
The Gauss hypergeometric function is defined as 

where (x) n stands for the Pochhammer symbol (x)k — x(x + 1) . . . (x + k — 1), 
with the convention (x)o = 1. When a = —n 6 —No, it is a polynomial 



2 



The following relations are useful: 



c 



is the confluent hypergeometric function. 
It satisfies Kummer's formula: 



" Z xF 1 ( a c - -z)= 1 F 1 ( C c a ; z). ( r,2i 



the recursion formula 



and the derivative formula 



(46) 



It is known that, uniformly for z in a compact set, for b, c fixed 
where 
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